Display device configured as an illumination source

ABSTRACT

The disclosed technology relates to an electronic device including a display device and an image sensor. The device may include a processor configured to cause the display device to output an illumination image in response to a command to capture one or more digital images of a scene. The processor can also be configured to cause the image sensor to receive the one or more digital images of the scene while the illumination image is displayed.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of, and claims priorityunder 35 U.S.C. § 120 to co-pending and commonly owned U.S. patentapplication Ser. No. 15/694,715 entitled “DISPLAY DEVICE CONFIGURED ASAN ILLUMINTION SOURCE” filed on Sep. 1, 2017, which is a continuationapplication of, and claims priority under 35 U.S.C. § 120 to commonlyowned U.S. patent application Ser. No. 15/362,595 entitled “DISPLAYDEVICE CONFIGURED AS AN ILLUMINATION SOURCE” filed on Nov. 28, 2016,which is a continuation application of, and claims priority under 35U.S.C. § 120 to commonly owned U.S. patent application Ser. No.13/932,844 entitled “DISPLAY DEVICE CONFIGURED AS AN ILLUMINATIONSOURCE” filed on Jul. 1, 2013, now granted as U.S. Pat. No. 9,525,811,the entireties of which are incorporated by reference herein.

TECHNICAL FIELD

The disclosed technology relates to an electronic device configured touse the device's own display to provide an illumination source forfront-facing image sensors. Aspects are also directed to methods ofusing the same.

DESCRIPTION OF THE RELATED ART

Many digital devices are equipped with a front-facing image sensor forcapturing self-images of a user. However, most of the devices equippedwith front-facing image sensors lack a dedicated illumination source toprovide additional illumination for capturing the self-image using thefront-facing image sensor in a low-light environment. In many cases, thebenefit of adding such an illumination source does not outweigh theadded technological complexity and the associated cost of having adedicated illumination source for the digital devices havingfront-facing image sensors.

SUMMARY

In one aspect, a mobile device for capturing one or more digital imagesof a subject is disclosed. The mobile device includes an image sensor, adisplay device, a memory, and a processor coupled to the memory. Theprocessor is configured to receive a command to capture the one or moredigital images, determine that the one or more digital images includethe subject, cause the display device to output, in a dynamicillumination mode and based on a preexisting illumination condition, anillumination image in response to the command to capture the one or moredigital images of the subject, and cause the image sensor to receive theone or more digital images of the subject while the subject isilluminated by the illumination image. The subject is not depicted bythe display device while the illumination image is output by the displaydevice.

In another aspect, a method is disclosed for capturing one or moredigital images of a subject using a mobile device including an imagesensor and a display device. The method includes receiving a command tocapture the one or more digital images, determining that the one or moredigital images include the subject, causing the display device to outputan illumination image, in a dynamic illumination mode and based on apreexisting illumination condition, an illumination image in response tothe command to capture the one or more digital images of the subject,wherein the subject is not depicted by the display device while theillumination image is output by the display device, and capturing theone or more digital images of the subject using the image sensor whilethe subject is illuminated by the illumination image.

In another aspect, a non-transitory computer-readable medium comprisinginstructions that when executed cause the mobile device, which includesan image sensor and a display device, to perform the following stepsincluding receiving a command to capture the one or more digital images,determining that the one or more digital images include the subject,causing the display device to output an illumination image, in a dynamicillumination mode and based on a preexisting illumination condition, anillumination image in response to the command to capture the one or moredigital images of the subject, wherein the subject is not depicted bythe display device while the illumination image is output by the displaydevice, and capturing the one or more digital images of the subjectusing the image sensor while the subject is illuminated by theillumination image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a perspective view that illustrates a typical digital devicewith a front-facing image sensor and a user using the digital device tocapture a self-image or a self-video according to one embodiment.

FIG. 1B is a perspective view that illustrates typical digital deviceswith front-facing image sensors and multiple users using the digitaldevices to exchange images or videos through a network according toanother embodiment.

FIG. 2 is a functional block diagram illustrating a digital devicecomprising a front-facing image sensor and a display device configuredas an illumination source according to one embodiment.

FIG. 3A is a flow chart illustrating a method of using a digital devicewith a front-facing image sensor and a display device configured as anillumination source according to one embodiment.

FIG. 3B is a flow chart illustrating a method of determining apreexisting illumination condition according of the embodiment of FIG.3A.

FIG. 3C is a flow chart illustrating a method of adjusting the displaydevice to an optimized imaging illumination according of the embodimentof FIG. 3A.

FIG. 3D is a flow chart illustrating a method of adjusting the displaydevice to a default imaging illumination according of the embodiment ofFIG. 3A.

FIGS. 4A-4L are block diagrams illustrating various embodiments ofillumination images displayed on the display device of a digital devicehaving a front-facing image sensor and a display device configured as anillumination source according to one embodiment.

DETAILED DESCRIPTION

Many digital devices come with a front-facing image sensor for capturingself-images a user. The captured self-image may be a static image suchas a photograph, or may be a dynamic image such as a video. However,most if not all devices with front-facing cameras lack a dedicatedillumination source (e.g., a flash or LED light for capturing stillimages or video). As a result, when using the front-facing image sensorin a low-light environment, the illumination from ambient light may notbe sufficient to provide adequate illumination for the image sensor.While adding a flash or an LED source can provide a solution, thebenefit of adding such an illumination source does not outweigh theadded technological complexity and the associated cost of having adedicated illumination source for a front-facing camera on the digitaldevices. Thus, there is a need for a cost-effective illumination sourcefor capturing images using the front-facing image sensor of a digitaldevice.

The disclosure is directed to an electronic device having a front-facingimage sensor and a digital display, where the electronic device isconfigured to use the digital display as an illumination source for thefront-facing image sensor. Aspects are also directed to methods of usingthe same. One advantage of the system described herein is that itimproves the low-light performance of the front-facing image sensor ofthe electronic device without incurring the added costs or complexity ofan additional illumination source.

Thus, one embodiment is an electronic device that is configured toilluminate the digital display as an image is being captured by a frontfacing camera. The user may activate the front facing camera to capturean image, and this would cause the digital display to flash a brightwhite color while the image is being captured. In another aspect, thedigital display may brighten to a predefined brightness, or to apredefined color, as the image is being captured. This feature may allowthe user to choose how the digital display is used to improve alow-light capture of images from the front camera.

The following disclosure may describe the features of variousembodiments of a digital device having a front-facing image sensor and adisplay device configured as an illumination source in the context ofone type of device (e.g., a smart phone). However, it is to beunderstood that other embodiments are possible, including any suitableelectronic devices that can be configured to have a front-facing imagesensor and a display device configured as an illumination source. Suchdevices include, for example, mobile phones, tablet computers, notebookcomputers, desktop computers, video cameras, portable music players,among others. In addition, the display device that may provide thisfunction include an LED, LCD, OLED, AMOLED, or other similar types ofdisplays that can be configured as an illumination source for afront-facing image sensor of a digital device.

FIG. 1A illustrates a digital device 102 with a front-facing imagesensor 110 and a user 120 using the digital device to capture aself-image or a self-video according to one embodiment. As discussedabove, the illustrated digital device 102 may be a tablet or a smartphone, although aspects are not limited thereto. The digital device 102includes a display device 104 that displays images of what is capturedthrough the front-facing image sensor 110. In the illustratedembodiment, the display device 104 is configured to display anillumination image 106, which includes a self-image of the user 120 inthis example. The self-image of the user as an illumination image 106captured by the front-facing image sensor 110 may be captured inresponse to a command by the user 120.

As described herein, an “image” may refer to not only a still digitalimage as but may also refer to a video comprising instantaneous framesof many images. In addition, an image can refer to images displayed onthe display device 104, or images that exist in a memory device orstorage device of the digital device 102 but not displayed on thedisplay device 104.

As shown, the user 120 would begin an image capture mode with thedigital device 102 wherein an illumination image 106 would be displayed.The user 120 could then activate a shutter button to capture the imageat a particular point. As the shutter button is activated, the digitaldevice 102 would instruct the display device 104 to flash a bright whitecolor that would better illuminate the user 120. This would improve theimage being captured by adding additional light onto the user 120.

FIG. 1B illustrates digital devices with front-facing image sensors andmultiple users using the digital devices to exchange images or videosthrough a network according to another embodiment. The digital device102A in use by a first user 120A includes a first front-facing imagesensor 110A and a first display device 104A configured to display afirst illumination image. In this example, the illumination imageincludes a self-image 106A of the first user 102A captured by the firstfront-facing image sensor 110A in response to a command by the firstuser 120A. Analogously, the digital device 102B in use by a second user120B includes a second front-facing image sensor 110B and a seconddisplay device 104B configured to display a second illumination image.In this example, the second illumination image includes a secondself-image 106B of the second user 102B captured by the secondfront-facing image sensor 110B in response to a command by the seconduser 120B.

During use, such as a video call, the first display device 104A may beconfigured to brighten as the user 120 a is in the call. Thisbrightening would allow the system to transmit a higher quality image tothe second user 120 b. Similarly, the second display device 104 b couldbe configured to brighten while the second user 120 b was on a videocall.

A digital device with a front-facing image sensor such as the first andsecond digital devices 102A and 102B of FIG. 1B can also be configuredto convert electrical signals generated by image sensors in response todetected photons into electromagnetic signals and transmit theelectromagnetic signals. The digital device can further be configured toreceive similar electromagnetic signals generated by another devicecommunicatively coupled to the digital device. For example, in theillustrated embodiment of FIG. 1B, the first digital device 102A may beconfigured to convert electrical signals corresponding to the firstself-image 106A, generated in response to photons detected by the firstimage sensor 110A, and convert the electrical signals into first uploadelectromagnetic signal 122A. The information contained in the firstupload electromagnetic signal 122A can in turn be received by the seconddigital device 102B configured to receive first download electromagneticsignal 122B through the network 130 and convert the first downloadelectromagnetic signal 122B into electrical signals, which is thendisplayed as a first communicated image 108B. The first communicatedimage 108B in this example corresponds to the first self-image 106Acaptured by the first image sensor 110A. Analogously, the second digitaldevice 102B can be configured to convert electrical signalscorresponding to the second self-image 106B, generated in response todetected photons by the second image sensor 110B, and convert theelectrical signals into second upload electromagnetic signal 124B. Theinformation contained in the second upload electromagnetic signal 124Bcan in turn be received by the first digital device 102A configured toreceive second download electromagnetic signal 124A through the network130 and convert the second download electromagnetic signal 124A intoelectrical signals, which is then displayed as a second communicatedimage 108A corresponding to the second self-image 106B captured by thesecond image sensor 110B.

FIG. 2 is a functional block diagram illustrating a digital device 200with a front-facing image sensor and a display device configured as anillumination source, such as a smart phone, according to one embodiment.The digital device 200 includes a command input module 210, anillumination sensing module 220, an illumination adjustment module 230,and a front-facing image sensor module 240. Each of the command inputmodule 210, the illumination sensing module 220, the illuminationadjustment module 230, and the front-facing image sensor module 240 arecommunicatively connected to a central processing module 250. Thedigital device 200 further includes a memory module 260 and a storagemodule 270 communicatively connected to the central processing module250. The digital device 200 further includes a communication subsystem280 configured to communicatively connect the digital device 200 to anetwork 290.

The illustrated digital device 200 includes the central processingmodule 250 configured to control the overall operation of the digitaldevice 200 and may include a suitable microprocessor configured toperform processing functions of the digital device 200. In someembodiments, the central processing module 250 includes specializedsub-processing modules such as a graphics processing module.

The digital device 200 further includes the command input module 210configured to receive various modes of command input from a user. Insome embodiments, the command input module 210 can include any number ofsuitable input devices such as a voice recognition device, a gesturerecognition device, a motion sensing device, a touch screen device, akeyboard device, and an auxiliary input/output (I/O) device, amongothers. The command input module can also include supporting circuitryto transform physical input signals such as a voice wave or a motioninto digital signals.

The digital device 200 further includes the illumination sensing module220 configured to determine an illumination condition. The illuminationsensing module 220 comprises the front-facing image sensor and an imagesensor controller. The image sensor includes a plurality of pixelsconfigured to convert incident photons into electrical signals, whichare transferred to the central processing module to be processed. In atypical image sensor, each pixel includes a photosensitive area, whichis configured to absorb incident photons of light. In some embodiments,incident photons may be directed by a micro lens over each pixel toenhance the quantum efficiency of photon collection. The absorbedphotons are converted into electrons, whose number may depend on theenergy of the incident photon. The electrons are in turn converted to avoltage signal.

In some embodiments, the image sensor includes a charge-coupled device(CCD) image sensor. A CCD image sensor comprises a color filter arrayand a pixel array. Each pixel of a CCD image sensor includes a colorfilter comprising a pattern of red, green and blue filters. In oneexample, the color filter may be arranged in a Bayer filter patternhaving a 2×2 checker board color filter pattern. The 2×2 checkerboardfilter pattern of a Bayer filter includes one red and one blue filtersdisposed diagonally to one another and two green filters disposeddiagonally to one another. The filtered photons passing throughdifferent color filters are then absorbed by a photodiode within thepixel array. The photodiode converts the absorbed photons into a charge,and the charge is moved to a single location by applying differentvoltages to pixels, in a process called charge-coupling. Because thecharge in the pixel is moved by applying different voltages, CCD imagesensors are supported by external voltage generators.

In some embodiments, the image sensor includes a complementary metaloxide semiconductor (CMOS) image sensor. Like CCD image sensors, CMOSimage sensors include an array of photo-sensitive diodes, one diodewithin each pixel. Unlike CCDs, however, each pixel in a CMOS imager hasits own individual integrated amplifier. In addition, each pixel in aCMOS imager can be read directly in an x-y coordinate system, ratherthan through movement of a charge. Thus, a CMOS image sensor pixeldetects a photon directly and converts it to a voltage, which isoutputted.

The illumination sensing module 220 includes additional circuitry forconverting the outputted voltages resulting from an incident photon intodigital information, which may be processed by the central processingmodule 250. The illumination sensing module 220 further includes animage sensor controller configured to control the image sensor inresponse to various commands from the central processing module 250.

The illumination adjustment module 230 may be configured to adjust theillumination conditions of the display device to and from an imagingillumination condition and a normal viewing illumination condition, inresponse to a command received from a user. The illumination adjustmentmodule includes the display device and a display controller. In oneembodiment, the display device can include an active matrix organiclight-emitting diode (AMOLED) display comprising an active matrix oforganic light-emitting diode (OLED) pixels that generate light uponelectrical activation. The OLED pixels can be integrated onto a thinfilm transistor (TFT) array, which functions as a series of switches tocontrol the current flowing to each individual pixel. Other embodimentsof the display device are possible, including an LED, LCD, OLED, AMOLED,or any other similar types of displays that can be configured as anillumination source for a front-facing image sensor of the digitaldevice 200.

The light emission intensity and therefore the luminance of each pixelwithin a display can be adjusted by the current supplied to a emittingelement, such as a light-emitting diode (LED). In one implementation,the display is an active matrix display such as an AMOLED, whose pixelscomprise two transistors and a capacitor. A first transistor whose drainis connected to a light emitting diode (e.g., OLED) is configured tocontrol the amount of current flowing through the diode and thereforethe light emission intensity by controlling a gate-source voltage of thefirst transistor. The gate-source voltage is in turn maintained by thecapacitor connected between the gate and the source of the firsttransistor. The gate-source voltage can be modified by controlling theamount of charge stored in the capacitor through controlling a secondtransistor, whose gate is connected to a row select line and whosesource is connected to a data line. Thus, by controlling variousvoltages such as the row select line voltage and the data line voltageto control the second transistor, which in turn controls the currentdelivered to the light emitting diode through the first transistor, theluminance value of each pixel in the display device can be adjusted toprovide varying degrees of illumination for the front-facing imagesensor.

The front-facing image sensor module 240 is configured to capture thedigital image through the front-facing image sensor under the imageillumination condition. The front-facing image sensor module can includeand share similar hardware devices as the illumination sensing module.For example, the front-facing image sensor module 240 comprises thefront-facing image sensor and an image sensor controller, whosefunctions and operations are substantially the same as the illuminationsensing module 220. In addition, the illumination adjustment moduleperforms calculations necessary to determine various illuminationconditions for the display device of the illumination adjustment module230.

The digital device 200 further includes the memory module 260 configuredto store information while the digital device 200 is powered on. Thememory module 260 can include memory devices such as a static randomaccess memory (SRAM) and a dynamic random access memory (RAM). Thememory devices can be configured as different levels of cache memorycommunicatively coupled to the central processing module 250 through amemory bus that provides a data path for flow of data to and from thememory devices and the microprocessor. In particular, the memory modulemay hold image information at various stages of the operation of thedigital device to provide illumination for the front-facing image sensorusing the display device.

The digital device 200 further includes the storage module 270configured to store media such as photo and video files, as well assoftware codes. In some embodiments, the storage module 270 isconfigured to permanently store media even when the digital device 200is powered off. In some implementations, the storage module 270 includesstorage media, such as a hard disk, a nonvolatile memory such as flashmemory, read-only memory (ROM), among others.

The digital device 200 further includes the communication subsystem 280configured to communicatively connect the digital device 200 to thenetwork 290. The communication subsystem 280 includes circuitryconfigured for wireless communication. For example, the communicationsubsystem 280 may enable Wi-Fi® communication between the digital device200 and the network 290 using one of 802.11 standards. The communicationsystem 280 may additionally enable standards such as BLUETOOTH®, CodeDivision Multiple Access® (CDMA), and Global System for MobileCommunication® (GSM), among others.

FIGS. 3A-3D are flow charts illustrating a method 300 of using a digitaldevice with a front-facing image sensor and a display device configuredas an illumination source according to one embodiment. The methodincludes receiving a command to capture the digital image, adjusting thedisplay device to an imaging illumination condition in response to thecommand, and capturing the digital image using the front-facing imagesensor under the imaging illumination condition.

The digital device of the illustrated embodiments in FIGS. 3A-3D may bea digital device such as the digital device 200 of FIG. 2 with afront-facing image sensor and a display device configured as anillumination source according to one embodiment.

The method 300 of using a digital device with a front-facing imagesensor and a display device configured as an illumination source beginsat a start state 310 and moves to a state 320 to receive a command tocapture a digital image using the front-facing image sensor. In oneaspect, the command may be received in any suitable form that can beprocessed by the command input module 210, including a voice commandprocessed by a voice recognition device, a gesture command processed bya gesture recognition device, a touch command processed by a touchscreen device, a keyboard command processed by a keyboard device, amotion command processed by a motion sensing device, among othersuitable forms of a user command.

After receiving the command to capture the digital image at the state320, the method 300 moves to a state 330 and activates the front-facingimage sensor. In one aspect, activating the front-facing image sensor atstate 330 can include, for example, providing an access voltage to theaccess lines of the image sensor and providing Vcc to an image sensorcontroller of the image sensor module 220.

An illumination condition provided by the display device can be definedby many parameters, including luminance and chrominance values of thepixels of the display device. For example, as a person having ordinaryskill in the art will understand, the actual values of luminance andchrominance depend on the color space being used to describe them. Forexample, in RGB or sRGB color spaces, each pixel can have a relativeluminance Y represented by the equation Y=rR+gG+bB, where R, G, and Brepresent color components red, green and blue and r, g, b areconstants. For example, for sRGB space, the constants r, b, and b havevalues 0.212, 0.7152, and 0.0722, respectively. In Y′UV color space, forexample, Y′ represents a luma value and U and V represent two colorcomponents. RGB space and the Y′UV space are related by the well-knowntransformational relationships:

$\begin{matrix}{\begin{bmatrix}Y^{\prime} \\U \\V\end{bmatrix} = {\begin{bmatrix}0.299 & 0.587 & 0.114 \\{- 0.14713} & {- 0.28886} & 0.436 \\0.615 & {- 0.51499} & {- 0.10001}\end{bmatrix}\begin{bmatrix}R \\G \\B\end{bmatrix}}} & (1) \\{\begin{bmatrix}R \\G \\B\end{bmatrix} = {\begin{bmatrix}1 & 0 & 1.13983 \\1 & {- 0.39465} & {- 0.58060} \\1 & 2.03211 & 0\end{bmatrix}\begin{bmatrix}Y^{\prime} \\U \\V\end{bmatrix}}} & (2)\end{matrix}$

In addition, a person skilled in the art will also understand that anysuitable color space representation, such as one of YUV, YCbCr, YPbPr,etc, can be used to represent an illumination condition of the pixels ofthe display device. In the description herein, the term “luminance” isused generally to refer to an overall intensity of the light, and theterm “chrominance” is used generally to refer to a color component.

According to one embodiment, the method 300 of using the digital devicewith a front-facing image sensor includes providing a dynamicillumination mode, which can be selected by the user. When activated bythe user, the dynamic illumination mode allows for an optimization ofthe illumination condition provided by the display device based on apre-existing illumination condition determined by the illuminationsensing module. When the dynamic illumination mode is not activated bythe user, a predetermined default illumination condition is provided bythe display device, irrespective of the pre-existing illuminationcondition. Details of the illumination modes will be more apparent inthe discussions that follow. After activating the front-facing imagesensor at the state 330, the method 300 moves to a decision state 340 todetermine whether or not a dynamic illumination mode has been activated.

When a determination is made at the decision state 340 that the dynamicillumination mode is not activated, the method 300 adjusts the displaydevice to a default imaging illumination condition at a process state350. Additional details on the steps performed to adjust the displaydevice at the state 350 are discussed below with reference to FIG. 3D.The method 300 then activates an image capture shutter at a state 390.

However, when a determination is made at the decision state 340 that thedynamic illumination mode is activated, the method 300 moves to aprocess state 360 to determine a preexisting illumination condition.Additional information on how to determine a preexisting illuminationcondition can be found with reference to FIG. 3B below.

Once the preexisting illumination condition has been determined at theprocess state 360, the method 300 moves to a decision state 370 todetermine whether additional illumination is needed. This determinationmay be based on the computed difference between an average luminancevalue of the subject and a stored luminance criteria corresponding tothat subject. If the computed difference exceeds a certain thresholdpercentage value, the method 300 may proceed to a process state 380 toadjust the display device to an optimized imaging illuminationcondition. However, if the computed difference does not exceed a certainthreshold percentage value, the method 300 proceeds to the process state350 to adjust the display device to a default imaging illuminationcondition as discussed above.

By way of an example only, the stored target luminance criteria for ahuman face may include 18% in gray scale of the luminance curve. In an8-bit luminance curve, there may be 2⁸=256 levels of luminance valuessuch that 18% in gray scale corresponds to 46^(th) gray level. If theaverage luminance value of the human face captured in the test frame hasan average luminance value corresponding to, for example, 10% in grayscale corresponding to 26^(th) gray level in an 8-bit luminance curve,the computed difference would be 8%. Whether the method 300 proceeds toadjusting the display device to an optimized imaging illuminationcondition or to adjusting the display device to a default imagingillumination condition may depend on whether or not the computeddifference of 8% exceeds the threshold value in one embodiment.

After adjusting the display device to an optimized imaging illuminationcondition at the process state 380, the method 300 moves to the state390 to activate the shutter. The method 300 then moves to a state 392wherein the image or the video frame is captured while the illuminationimage is displayed on the display device. The method 300 then moves to astate 394 wherein the shutter is deactivated. Finally, the method 300moves to a state 396 wherein the display device is returned to normalillumination condition.

FIG. 3B is a flow chart which provides additional details on the process360 for determining a preexisting illumination condition according toone implementation discussed above in connection with FIG. 3A. Theprocess 360 includes capturing a test frame using the front-facing imagesensor at a state 362 and computing a difference between an averageluminance value of the test frame and a stored luminance criteria at astate 366. In addition, in another implementation, the process 360 fordetermining a preexisting illumination condition can further includedetermining a subject in the test frame at a state 364. In thisimplementation, computing the difference at the state 366 includescomputing a difference between an average luminance value of a subjectof the test frame and a stored luminance criteria corresponding to thesubject. For example, the subject can include a face, multiple faces, abody, multiple bodies, and a landscape, among others. Additional detailsof the states 362, 364, and 366 are discussed below.

According to one implementation, the process 360 for determining thepreexisting illumination condition includes capturing a test frame atthe state 362. A test frame may be a frame captured using a set of fixedtest frame imaging conditions, including an f-number and an exposuretime. In some implementations, the test frame imaging conditions includea relatively low f-number and a relatively short exposure time comparedto actual imaging conditions in order to maximize speed. In otherimplementations, the test frame imaging conditions includes an f-numberand an exposure time that are similar to actual imaging conditions.

Still referring to FIG. 3B, process 360 for determining the preexistingillumination condition according to one implementation further includesdetermining a subject in the test frame at the state 364. In one aspect,determining the subject can include determining a metering region anddetermining the subject to be imaged based on the information gatheredfrom the metering region.

Determining the metering region may include determining a rectangulararea comprising a fixed percentage of the total display area of the testframe to be the metering region. By way of an example only, the meteringregion may have, for example, a rectangular metering region having awidth equal to about 75% of the test frame width and a length equal toabout 75% of the test frame length. Other embodiments are possible,where the metering region may include a non-rectangular area and/or arectangular area occupying different percentages of the length and/orwidth of the test frame.

In another aspect, determining the subject to be imaged may be based onany suitable number of subject determination criteria. In someimplementations, the subject determination criteria may includedetermining a fraction of the total test frame area a potential subjectoccupies. In other implementations, the subject determination criteriamay include an average luminance value of the potential subject comparedto an overall average of luminance of the total test frame. In yet otherimplementations, the subject determination criteria may include othercriteria such as an average value of the color components of thepotential subject compared to an average value of the color componentsof the total test frame. Using one or more of the subject determinationcriteria and comparing against a reference list stored in the storagemodule, a subject of the test frame can be determined.

In another aspect, determining the subject to be imaged may includedetermining that the subject includes a human face. Determining that thesubject is a human may invoke any one or more of face-detectionalgorithms known in the art. For example, the determination of a humanface can be made based on any number of suitable factors, such as theovular nature of the subject and minimum and maximum distances betweenthe center point and the outer boundaries of the subject.

Still referring to FIG. 3B, the process 360 for determining thepreexisting illumination condition according to one implementationincludes computing a difference between the average luminance value ofthe subject of the test frame and a stored luminance criteriacorresponding to that subject at the state 366.

FIG. 3C illustrates a flow chart which provides additional details onthe process 380 for adjusting the display device to an optimized imagingillumination condition discussed above in connection with FIG. 3A. Theprocess 380 begins at a state 382 by calculating an additionalillumination based on the computed difference between the averageluminance value of subject and the stored luminance criteriacorresponding to the subject. The state 382 is followed by selecting anillumination image at a state 384, followed by adjusting an averageluminance value at a state 386, followed by displaying the illuminationimage at a state 388. Each of the states 382, 384, 386, and 388 arediscussed more in detail below.

In some embodiments, the calculated additional illumination in the state382 may be linearly or nonlinearly proportional to the computeddifference between the average luminance value of the subject and thestored luminance criteria corresponding to that subject in the state 366in FIG. 3B. The calculated additional illumination to be provided by thedisplay device may be a value obtained by, for example, multiplying thecomputed difference between the average luminance value of the subjectand the stored luminance criteria corresponding to that subject by otherfactors. One such factor may be a distance factor, for example, toaccount for the fact that a substantial amount of light intensity can bereduced as a function of the distance between the display device and thesubject being imaged.

In other embodiments, the additional illumination may be calculatedbased on a difference between an average chrominance value of thesubject and the stored chrominance criteria corresponding to thatsubject in the state 382. In this embodiment, color components havingrelatively low average values in the subject of the test frame may becalculated to be over-compensated by the display device while othercolor components having relatively high average values in the subject ofthe test frame may be calculated to be under-compensated so as topreferentially compensate color components in order to produce anaesthetically more pleasing image.

Still referring to FIG. 3C, the process 380 for adjusting the displaydevice additionally includes selecting an illumination image at thestate 384. The selected illumination image may be any suitable image forproviding the desired illumination for the front-facing image sensor.

In one implementation, the illumination image may be an image that wasbeing displayed before receiving the command to capture the digitalimage in state 320 in FIG. 3A, such as a default screen of the devicewith optimized luminance. In another implementation, the illuminationimage may be an image that was being displayed immediately prior toadjusting the display device, such as a preview frame of the user'simage captured by the front-facing image sensor with optimizedluminance. In yet another implementation, the illumination image may bean image having illumination regions configured such that the pixelsincluded in the illumination regions have optimized luminance and/orchrominance. Various configurations of the illumination image that canbe included in selecting the illumination image in the state 384 arediscussed more in detail below in connection with FIGS. 4A-4L.

The illumination image may be selected in the state 384 based on theadditional illumination calculated in the state 382. For example, asuitable illumination image may be the one capable of providing thecalculated additional illumination at the state 382. However, not allavailable illumination images may be capable of providing the calculatedadditional illumination at the state 382. As an illustrative example, afirst illumination image may have pixels arranged to provide 1-5% ofadditional luminance, whereas a second illumination image may havepixels arranged to provide 5-10% of additional luminance. In thisillustrative example, if the required additional luminance based on thecalculated additional image at the state 382 exceeds 5%, the secondillumination image would be selected over the first illumination imageat the state 384.

Still referring FIG. 3C, the process 380 for adjusting the displaydevice to an optimized imaging illumination condition further includesadjusting an average luminance value of the selected illumination imageto a target imaging luminance value at a state 386. Adjusting theaverage luminance value at the state 386 includes first determining adifference between an average luminance value of the selectedillumination image and a target imaging luminance value. Upondetermination of the difference, adjusting the average luminance valueat the state 386 further includes determining the voltages and currentsrequired for the pixels included in the illumination image to displaythe selected illumination image under the target default imagingluminance value.

Still referring FIG. 3C, the process 380 of adjusting the display deviceto an optimized imaging illumination condition further includesdisplaying the selected illumination image having the imaging luminancevalue at a state 388. Displaying the selected illumination image at thestate 388 includes selecting the pixels of the display devicecorresponding to the selected illumination image and supplying thevoltages and currents determined based on the difference between anaverage luminance value of the selected illumination image and thetarget imaging luminance value.

Referring now to FIG. 3D, the process 350 of adjusting the displaydevice to a default imaging illumination condition is explained in moredetail. The process 350 begins at a state 352 to select an illuminationimage, followed by a state 354 where an average luminance value of theillumination image is adjusted to a default imaging luminance value. Thestate 354 is in turn followed by displaying the illumination imagehaving the default imaging luminance value at a state 356. Each of thestates 352, 354, and 356 are described more in detail below.

Referring to FIG. 3D, the process 350 of adjusting the display device toa default imaging illumination condition includes selecting at the state352 an illumination image. Selecting at the state 352 an illuminationimage can include selecting a default illumination image. The defaultillumination image may be any suitable image for providing illuminationfor the front-facing image sensor. In one implementation, the defaultillumination image may be an image that was being displayed beforereceiving the command to capture the digital image at the state 320 inFIG. 3A. For example, the default illumination image may be any one ofdefault screens of the display device that may include, for example,application icons. In another implementation, the default illuminationimage may be an image that was being displayed immediately prior toadjusting the display device. For example, the default image may be apreview frame of the user's image captured by the front-facing imagesensor prior to capturing a permanent image. In another implementation,the default illumination image may be an image having an illuminationarea configured such that the pixels included in the illumination areahave a predetermined luminance value. In some embodiments, the defaultillumination image can be one of predetermined images stored by thedevice manufacturer in the storage module of the digital device. Inother embodiments, the default illumination image can be provided by theuser and stored in the storage module. For example, the image can be anyimage stored in the storage module by the user, such a personalportrait, a web page, among others. Various configurations of theillumination image that can be included in selecting at the state 352the illumination image are discussed more in detail below in connectionwith FIGS. 4A-4L.

Still referring FIG. 3D, the process 350 of adjusting the display deviceto a default imaging illumination condition further includes adjustingat the state 354 an average luminance value of the default illuminationimage to a default imaging luminance value. Adjusting at the state 354the average luminance value includes first determining a differencebetween an average luminance value of the selected default illuminationimage and a target default imaging luminance value. Upon determinationof the difference, adjusting the average luminance value at the state354 further includes determining the voltages and currents required forthe pixels included in the illumination image to display the defaultillumination image under the target default imaging luminance value.

Still referring FIG. 3D, the process 350 of adjusting the display deviceto a default imaging illumination condition further includes displayingat the state 356 the default illumination image having the defaultimaging luminance value. Displaying the default illumination image atthe state 356 includes selecting the pixels of the display devicecorresponding to the default illumination image and supplying thevoltages and currents determined based on the difference between anaverage luminance value of the selected default illumination image andthe target default imaging luminance value.

FIGS. 4A-4L illustrate exemplary implementations of illumination imagesselected in adjusting 380 the display device to an optimized imagingillumination condition and adjusting 350 the display device to a defaultimaging illumination condition. Each of FIGS. 4A-4L depicts an exemplaryimplementation of a digital device 400 comprising a front-facing imagesensor 402. Although the digital device 400 depicted in FIGS. 4A-4L is asmart phone, the digital device 400 can be any one of a mobile phone, atablet computer, a notebook computer, a desktop computer, a videocamera, a portable music player, among other digital devices that can beconfigure to include a front-facing image sensor. In addition, in eachof the embodiments illustrated in FIGS. 4A-4L, the digital device 400 isconfigured to display an illumination image 404. It is to be understoodthat each of the implementations illustrated in FIGS. 4A-4L, or anyfeatures included in the implementations, can be combined to formembodiments not depicted in FIGS. 4A-4L. In addition, the number, theshapes, and the physical sizes of different features are provided asexamples only, and other embodiments having a different number,different shapes, and different physical sizes are possible.

The illumination image 404 may be any suitable image displayed on thedisplay device of the digital device 400 for providing adequateillumination for the front-facing image sensor 402. In someimplementations, the illumination image 404 may be an image that wasbeing displayed prior to receiving a command from the user to capture adigital image. One implementation of such an illumination image isdepicted in FIG. 4A. The illumination image 404 of FIG. 4A is that of adefault image 406. The default image 406 may include visual andinteractive features such as a clock, a search window, applicationicons, among other features. In this implementation, upon receiving thecommand to capture a digital image, an average luminance value of thedefault image 406 may be adjusted according to adjusting 380 the displaydevice to an optimized imaging illumination or according to adjusting350 the display device to a default imaging illumination conditiondiscussed above in connection with FIG. 3A.

In some implementations, the illumination image may include one or moreillumination regions configured such that pixels included in theillumination regions are configured to illuminate white light. A pixelmay be configured to illuminate white light when the intensities ofindividual color components (e.g., R, G, and B of RGB color space) arebalanced to have substantially the same values such that a human eyeperceives the resulting light as being neutral without having a colorpreference.

The illumination image 404 of the digital device 400 in FIG. 4Baccording to one implementation includes a single illumination region408 that is configured to illuminate white light and substantially coverthe entire illumination image 404. In this implementation, as in FIG.4A, an average luminance value of the illumination region 408 may beadjusted according to adjusting 380 the display device to an optimizedimaging illumination or according to adjusting 350 the display device toa default imaging illumination condition discussed above in.

The illumination image 404 of the digital device 400 in FIG. 4Caccording to another implementation includes a single illuminationregion 410 that is configured to illuminate white light and cover aportion of the illumination image 404. An average luminance value of theillumination region 410 may be adjusted as in FIGS. 4A-4B. In addition,the size of the illumination region 410 may be adjusted to increase thenumber of pixels included in the illumination region 410 to increase theoverall illumination of the display device. Pixels included in a passiveregion 412 are configured to have negligible luminance values.

The illumination image 404 of the digital device 400 in FIG. 4Daccording to another implementation includes a plurality of illuminationregions 416 a-416 c, each of which is configured to illuminate whitelight and cover a portion of the illumination image 404. Averageluminance value of the illumination regions 416 a-416 c may be adjustedas in FIGS. 4A-4C. In this implementation, the average luminance valuesof the illumination regions 416 a-416 c may be adjusted individually orcollectively. In addition, the sizes of the illumination regions 416a-416 c may be adjusted individually or collectively to optimize theoverall illumination of the display device. Pixels included in passiveregions 414 a-414 c outside of the illumination regions 416 a-416 c areconfigured to have negligible luminance values.

In some implementations, the illumination image 404 may include one ormore illumination regions configured such that pixels included in theillumination regions are configured to preferentially illuminate coloredlight of a color component (e.g., R, G, or B in RGB space). The pixelsmay be configured to preferentially illuminate colored light when theintensity of one of the color components is enhanced while intensitiesof other color components are suppressed such that a human eye perceivesthe resulting light as having a color. For example, to preferentiallyilluminate red light, the photodiodes corresponding to green and bluelights may be suppressed such that the color component R has arelatively high value in comparison to the color components G and B.

The illumination image 404 of the digital device 400 in FIG. 4Eaccording to one implementation includes a plurality of illuminationregions, where each of the illumination regions is configured topreferentially illuminate a colored light and to cover a portion of theillumination image 404. In the illustrated implementation, theillumination image 404 includes three illumination regions 420, 422, and424, each of which are configured to preferentially illuminate red,green, or blue light, respectively. An average luminance value of eachof the individual illumination regions 420, 422, and 424 may be adjustedas in FIGS. 4A-D. By individually adjusting the average luminance valueof each of the three illumination regions, whose values are derivedpreferentially weighed by the color component being preferentiallyilluminated, a combined illumination having a customizable color mix canbe provided for the front-facing camera 402. Pixels included in thepassive region 412 are configured to have negligible luminance values.

The illumination image 404 of the digital device 400 in FIG. 4Faccording to one implementation includes a plurality of illuminationregions, where the illumination regions configured to preferentiallyilluminate different colored light are interlaced and to coversubstantially the entire illumination image 404. In the illustratedimplementation, the illumination image 404 includes four illuminationregions 430, 432, 446, and 448, each of which are configured topreferentially illuminate red light, four illumination regions 432, 440,442, and 450, each of which are configured to preferentially illuminategreen light, and four illumination regions 434, 436, 444, and 452, eachof which are configured to preferentially illuminate blue light. Anaverage luminance value of each of the individual illumination regions430, 432, 434, 436, 438, 440, 442, 444, 446, 448, 450, and 452 may beadjusted as in FIGS. 4A-4E. As in FIG. 4E, by individually adjusting theaverage luminance value of each of the illumination regions, whosevalues are derived preferentially weighed by the color component beingpreferentially illuminated, a combined illumination having acustomizable color mix can be provided for the front-facing camera 402.In addition, the interlaced pattern may provide a more even distributionof colored light so as to avoid, for example, different portions of thesubject (e.g., a face) being preferentially illuminated with aparticular color component

The illumination image 404 of the digital device 400 in FIG. 4Gaccording to one implementation includes a plurality of illuminationregions, where the illumination regions configured to preferentiallyilluminate different colored light are distributed within theillumination image 404 and to cover at least a portion of theillumination image 404. In the illustrated implementation, theillumination image 404 includes three illumination regions 460 and 466,each of which is configured to preferentially illuminate red light, twoillumination regions 462 and 469, each of which are configured topreferentially illuminate green light, and two illumination regions 464and 470, each of which are configured to preferentially illuminate bluelight. An average luminance value of each of the individual illuminationregions 460, 462, 464, 466, 468, and 470 may be adjusted as in FIGS.4A-4F. As in FIG. 4E, by individually adjusting the average luminancevalue of each of the illumination regions, whose values are derivedpreferentially weighed by the color component being preferentiallyilluminated, a combined illumination having a customizable color mix canbe provided for the front-facing camera 402. In addition, the interlacedpattern may provide a non-uniform distribution of colored light so as toprovide, for example, different amounts of colored light to differentportions of the subject (e.g., a face). Pixels included in the passiveregion 412 are configured to have negligible luminance values.

The illumination image 404 of the digital device 400 in FIG. 4Haccording to one implementation includes a plurality of illuminationregions, where some of the illumination regions are configured topreferentially illuminate a colored light while other illuminationregions are configured to illuminate white light. In the illustratedimplementation, the illumination image 404 includes three illuminationregions 472, 474, configured to preferentially illuminate red, green, orblue light, respectively. The illumination image 404 additionallyincludes an illumination region 478 configured to illuminate whitelight. Average luminance values of each of the individual illuminationregions 472. 474, 476, and 478 may be adjusted as in FIGS. 4A-4G. Byindividually adjusting the average luminance value of each of theillumination regions 472, 474, and 476, whose values are derivedpreferentially weighed by the color component being preferentiallyilluminated, a combined illumination having a customizable color mix canbe provided for the front-facing camera 402. In addition, by adjustingthe average luminance value of the illumination region 478, additionalwhite light may be provided to increase the general brightness of thecaptured image. Pixels included in passive region 412 are configured tohave negligible luminance values.

In some implementations, the illumination image may include an imagecaptured by the front-facing image sensor. In some implementations, theimage captured by the front-facing image sensor may be a preview imagefor a still image. In other implementations, the image captured by thefront-facing camera may be a real-time frame being captured in a video.

One implementation using the image captured by the front-facing cameraitself as an illumination image is depicted in FIG. 4I. The illuminationimage 404 of FIG. 4I includes an illumination region 482 coveringsubstantially the entire illumination image 404 and includes an image ofa user 480 against a background. An average luminance value of theentire illumination image 404 may be adjusted as in FIGS. 4A-4H.

Another implementation using the image captured by the front-facingcamera itself as an illumination image is depicted in FIG. 4J. Inaddition to an illumination region 482 covering a portion of theillumination image 404 comprising an image of a user, the illuminationimage 404 of FIG. 4J additionally includes an illumination region 484covering a portion of the illumination image 404 and configured toilluminate white light. An average luminance value of each of theillumination regions 484 and 482 may be adjusted as in FIGS. 4A-4H.Pixels included in the passive region 412 are configured to havenegligible luminance values.

Another implementation using the image captured by the front-facingcamera itself as an illumination image is depicted in FIGS. 4K and 4L.In both implementations, in addition to an illumination region 482covering a portion of the illumination image 404 comprising an image ofa user, the illumination images 404 of FIGS. 4K and 4L additionallyincludes illumination regions 488 and 490 comprising an imagetransmitted from another device (e.g., of a second user 486) through thenetwork. An average luminance value of each of the illumination regions482, 488 and 490 may be adjusted as in FIGS. 4A-4J. Pixels included inthe passive region 412 in FIG. 4L are configured to have negligibleluminance values.

The previous description of the disclosed implementations is provided toenable any person skilled in the art to make or use the presentinvention. Various modifications to these implementations will bereadily apparent to those skilled in the art, and the generic principlesdefined herein may be applied to other implementations without departingfrom the spirit or scope of the invention. Thus, the present inventionis not intended to be limited to the implementations shown herein but isto be accorded the widest scope consistent with the principles and novelfeatures disclosed herein.

Clarifications Regarding Terminology

Implementations disclosed herein provide systems, methods and apparatusfor using the device's own display to provide an illumination source forfront-facing image sensors. One skilled in the art will recognize thatthese embodiments may be implemented in hardware, software, firmware, orany combination thereof.

In the description, specific details are given to provide a thoroughunderstanding of the examples. However, it will be understood by one ofordinary skill in the art that the examples may be practiced withoutthese specific details. For example, electrical components/devices maybe shown in block diagrams in order not to obscure the examples inunnecessary detail. In other instances, such components, otherstructures and techniques may be shown in detail to further explain theexamples.

Headings are included herein for reference and to aid in locatingvarious sections. These headings are not intended to limit the scope ofthe concepts described with respect thereto. Such concepts may haveapplicability throughout the entire specification.

It is also noted that the examples may be described as a process, whichis depicted as a flowchart, a flow diagram, a finite state diagram, astructure diagram, or a block diagram. Although a flowchart may describethe operations as a sequential process, many of the operations can beperformed in parallel, or concurrently, and the process can be repeated.In addition, the order of the operations may be re-arranged. A processis terminated when its operations are completed. A process maycorrespond to a method, a function, a procedure, a subroutine, asubprogram, etc. When a process corresponds to a software function, itstermination corresponds to a return of the function to the callingfunction or the main function.

The previous description of the disclosed implementations is provided toenable any person skilled in the art to make or use the presentinvention. Various modifications to these implementations will bereadily apparent to those skilled in the art, and the generic principlesdefined herein may be applied to other implementations without departingfrom the spirit or scope of the invention. Thus, the present inventionis not intended to be limited to the implementations shown herein but isto be accorded the widest scope consistent with the principles and novelfeatures disclosed herein.

What is claimed is:
 1. A mobile device for capturing one or more digitalimages of a subject, the mobile device comprising: an image sensor; adisplay device; a memory; and a processor coupled to the memory, theprocessor being configured to: receive a command to capture the one ormore digital images; determine that the one or more digital imagesinclude the subject; cause the display device to output, in a dynamicillumination mode and based on a preexisting illumination condition, anillumination image in response to the command to capture the one or moredigital images of the subject, wherein the subject is not depicted bythe display device while the illumination image is output by the displaydevice; and cause the image sensor to receive the one or more digitalimages of the subject while the subject is illuminated by theillumination image.
 2. The mobile device of claim 1, wherein theprocessor is further configured to: invoke a face-detection algorithm todetermine that the one or more digital images include the subject. 3.The mobile device of claim 1, wherein the illumination image is based onan even distribution of colored light.
 4. The mobile device of claim 3,wherein the illumination image comprises a white color.
 5. The mobiledevice of claim 1, wherein the command comprises a gesture or a motionof the subject.
 6. The mobile device of claim 1, wherein the processoris further configured to: determine an intensity of the illuminationimage based on a distance between the subject and the mobile device. 7.The mobile device of claim 1, wherein the subject comprises a face of aperson and the illumination image is based on the face of the persondepicted in a test image.
 8. The mobile device of claim 1, wherein theillumination image is based on a comparison between a chrominance valueof a test image and a target chrominance value.
 9. The mobile device ofclaim 8, wherein the target chrominance value is associated with thesubject, and wherein the subject comprises a face of a person.
 10. Themobile device of claim 9, wherein the one or more images are stilldigital images, wherein the illumination image consists of a singleregion displayed across the entirety of the display device, wherein thedisplay device comprises a touch screen, and wherein the commandcomprises a touch input.
 11. A method for capturing one or more digitalimages of a subject using a mobile device including an image sensor anda display device, the method comprising: receiving a command to capturethe one or more digital images; determining that the one or more digitalimages include the subject; causing the display device to output, in adynamic illumination mode and based on a preexisting illuminationcondition, an illumination image in response to the command to capturethe one or more digital images of the subject, wherein the subject isnot depicted by the display device while the illumination image isoutput by the display device; and capturing the one or more digitalimages of the subject using the image sensor while the subject isilluminated by the illumination image.
 12. The method of claim 11,wherein a face-detection algorithm is used to determine that the one ormore digital images include the subject.
 13. The method of claim 11,wherein the illumination image is based on an even distribution ofcolored light.
 14. The method of claim 13, wherein the illuminationimage comprises a white color.
 15. The method of claim 11, wherein thecommand comprises a gesture or a motion of the subject.
 16. The methodof claim 11, further comprising: determining an intensity of theillumination image based on a distance between the subject and themobile device.
 17. The method of claim 11, wherein the subject comprisesa face of a person and the illumination image is based on the face ofthe person depicted in a test image.
 18. The method of claim 11, whereinthe illumination image is based on a comparison between a chrominancevalue of a test image and a target chrominance value.
 19. The method ofclaim 18, wherein the target chrominance value is associated with thesubject, and wherein the subject comprises a face of a person.
 20. Themethod of claim 19, wherein the one or more images are still digitalimages, wherein the illumination image includes a single regiondisplayed across the entirety of the display device, wherein the displaydevice comprises a touch screen, and wherein the command comprises atouch input.
 21. A non-transitory computer-readable medium configured tostore instructions that, when executed by a processor of a mobile deviceincluding an image sensor and a display device, cause the mobile deviceto perform operations comprising: receiving a command to capture one ormore digital images; determining that the one or more digital imagesinclude a subject; causing the display device to output, in a dynamicillumination mode and based on a preexisting illumination condition, anillumination image in response to the command to capture the one or moredigital images of the subject, wherein the subject is not depicted bythe display device while the illumination image is output by the displaydevice; and capturing the one or more digital images of the subjectusing the image sensor while the subject is illuminated by theillumination image.
 22. The non-transitory computer-readable medium ofclaim 21, wherein a face-detection algorithm is used to determine thatthe one or more digital images include the subject.
 23. Thenon-transitory computer-readable medium of claim 21, wherein theillumination image is based on an even distribution of colored light.24. The non-transitory computer-readable medium of claim 23, wherein theillumination image comprises a white color.
 25. The non-transitorycomputer-readable medium of claim 21, wherein the command comprises agesture or a motion of the subject.
 26. The non-transitorycomputer-readable medium of claim 21, wherein execution of theinstructions causes the mobile device to perform operations furthercomprising: determining an intensity of the illumination image based ona distance between the subject and the mobile device.
 27. Thenon-transitory computer-readable medium of claim 21, wherein the subjectcomprises a face of a person and the illumination image is based on theface of the person depicted in a test image.
 28. The non-transitorycomputer-readable medium of claim 21, wherein the illumination image isbased on a comparison between a chrominance value of a test image and atarget chrominance value.
 29. The non-transitory computer-readablemedium of claim 28, wherein the target chrominance value is associatedwith the subject, and wherein the subject comprises a face of a person.30. The non-transitory computer-readable medium of claim 29, wherein theone or more images are still digital images, wherein the illuminationimage includes a single region displayed across the entirety of thedisplay device, wherein the display device comprises a touch screen, andwherein the command comprises a touch input.